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ABSTRACT 



Tne school counselor is forced to make important 
decisions in predicting student success. He makes these decisions, 
correctly or incorrectly, based on available data pertaining to that 
student. This two part study sought to determine the advantiges of 
using a computer for handling information and making predictions and 
to see whether the counselor would properly challenge computer 
predictions. Thus, ten junior high counselors supplied with 
corresponding computer predictions were asked to predict success in 
algebra for 100 anonymous eighth-grade students. Multiple linear 
regression analysis revealed that counselors tend to be optimistic in 
their predictions but that they were not unduly influenced by the 
computer predictions. A related document is available as ED 017 710. 
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SUjgjW 

The purpose of the project vas to continue and complete the development and 
evaluation of the man-machine counseling system that was designed as part of the 
USOE- supported Project No. l4l-65, "Exploratory Study of Information Processing 
and Computer Technology in Vocational Counseling. " The computer system vas 
designed for on-line use by secondary school counselors and would have operated 
on a large, time-sharing computer. Counselors would coranunicate with the system 
through teletypes located in the schools. Some major parts of the system were: 

1. Storage and retrieval of student Information. 

2. Calculation routines. 

3* Prediction routines 

4. Automatic tracking and monitoring of student progress. 

5. Automated interviews. 

6. Automated individualized letters or reports to students. 

Counselors at tvo Los Angeles schools had been intimately involved in the design 
phases of this system and were to be Involved in further development, installation 
evaluation and revision. They were also engaged in training in group and 
individual counseling techniques and other activities intended to prevent their 
becoming too machine or data oriented. Funding for this project was discontinued, 
however, before the work could be completed. 

Those parts of the computer system which had been programmed prior to the 
termination of the project encouraged belief that the system would have been 
powerful and useful, even for naive users. 

I. INTRODUCTION 

The purpose of this study vas to continue and complete the development and 
evaluation of the man-machine counseling system designed as part of the USOE- 
supported Project No. lUl-65; entitled "Exploratory Study of Information 
Processing and Computer Technology in Vocational Counseling. " Originally planned 
as a 28-month project, the major goals of the study had to be abandoned as the 
result of funding reductions at the U. S. Office of Education. In order to make 
use of what funds did remain, we proposed to USOE that we modify our plans to 
allow us to do a useful piece of work with the remaining funds. The modified 
plan was for an investigation of the effects of computer prediction on counselor > 
prediction of student success. The proposal was accepted and the investigation 
was conducted. It is described in another document, "Computer Prediction and 
Counselor Prediction," TM-4410. The purpose of this report is to describe the 
state of the project at the point when it was terminated. Most of the work on 
this project was conducted under a previous contract. For a detailed descrip- 
tion of that effort, the reader is referred to two System Development Corporation 
documents: IM-2582/OOO/0O and TM-3718. 

II. COMPUTER SYSTEM OBJECTIVES 

The general objective was to develop and evaluate a man- machine counseling system 
in a junior high school and a high school in Los Angeles, California. The 
computer system was to be designed for on-line use by counselors, and would 
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operate on a large time -shared computer. Counselors would communicate with the 
system through teletypes located at the schools. Although the teletype would be 
the customary mode of input and output, card readers and line printers would be 
used for bulk input or output when desired. The following computer functions 
were to be implemented: 

1. Student Information Storage and Retrieval 

Capability was to be provided for on-line (or off-line) data description, storage, 
and retrieval, 

a. Data Description . Hie system would set up data bases of items as named 
and described by the user, and new items could be named, described, and 
added to existing data bases constructed by the system. 

b. Data Storage . Data-storage statements prepare the program for the input 
of data into the data base. They specify the students and name the items 
to be stored. Input data will be checked against the data description 
and faulty inputs will be reported. 

c. Data Retrieval . The user would employ data retrieval statements to 
specify the data desired and the population of students whose records 
should be searched. Such requests would be made either by teletype or 
by punched card. Hie retrieved data may be output on either the teletype 
or the high-speed printer# Retrieval statements could be used to inspect 
data in the data base, or in conjunction with calculation functions and 
form descriptions (see below) to generate reports. 

d. Input and Output Forms . At certain times it would be necessary to read 

in or print out large amounts of data using certain prespecified formats. 
Some cases of input are: initial loading of the data base from cards; 

recording students* marks for the 15th-week progress reports; teachers* 
recommendations of candidates for reading classes. Some output examples 
are progress reports, program cards, and the total cum. 

2. Define Statements 

A define statement is a means whereby the user may make certain kinds of changes 
in the language in order to simplify frequently used expressions, to satisfy 
idiosyncratic desires, or to make the language more English-like (and, hence, 
easier to teach and use). It is a string-substitution device that, in conjunction 
with the structure of the basic language, can make a great contribution to ease 
of use. 

3 . Calculation Routines 

The counselor would be able to use the computer to perform arithmetic and 
logical operations on data in the data base, or as a very powerful "desk 
calculator." Calculation phrases may be used within retrieval statements to 
produce "derived" data. 
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4. Prediction Routines 

Part of a counselor’s job is anticipating what might or might not happen to his 
students at some future time. This, of course, is a prediction problem. 

However, not all counselors are well versed in prediction and statistics. 
Prediction routines would be provided to satisfy the needs of those counselors 
who do not remember their statistics courses, as well as for those who do. In 
order to predict, it is only necessary that the user specify the criterion 
variable, the predictor variables, and a population. A prediction formula 
would then be computed and retained for later use. Prediction formulas may be 
called upon by name for application to individual students. 

5. Tracking and Monitoring 

Tracking and monitoring is a special case of retrieval. In general, these 
routines would be used for information about situations that (l) occur 
infrequently, (2) have potentially serious consequences, and (3) may pass 
unnoticed until they result in disaster for a student. A tracking and 
monitoring statement is, basically, a retrieval statement that continues to 
remain in force until it is deleted. Any time there are data satisfying the 
retrieval request, they will be reported automatically. 

6. Reminder Secretary 

The reminder secretary routine would permit the counselor to store messages to 
be typed out for him at a later time of his choosing. 

7. Automated "Interviews ” 

Automated "interviews” for student use were to be developed within the PIANIT 
(Prc granming IANg uage for Interactive Teaching) system. 1 Because the number of 
terminals to be installed at each school was small, few students could have been 
accommodated and consequently the use of the interviews was to be experimental 
only. PIANIT was also to have been used to prepare individualized letters or 
reports for students. Because this use does not require student interaction at 
a terminal, it was not to be r^tricted to experimental explorations. 

III. HUMANISTIC OBJECTIVES 

The computer system was to be used by people in a strongly interactive mode. 
Hierefore, it was important to pay attention to the needs, desires, and 
limitations of the intended users while designing, implementing, and revising 
the system. Consequently, there was a set of humanistic objectives to guide 
development of the system: 

1. The computer system should be implemented with the individual user 
(counselor) in mind. Hie language and procedures provided for 



^Teingold, S. L., and Frye, C. H. User’s Guide to PIANIT. SDC document 
TM-3055/OOO/OI . October 17, 1966. 214 pp. 
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communication between the user and the system should be as natural as 
possible, so that they would be easy to team and easy to use. 
Furthermore, they should be modifiable by the user to meet his 
individual needs. 

2. The counselors should be made aware of the possible dangers involved 
in using computers in the counseling operation, including such 
problems as safegarding the privacy of information, overvaluation of 
computer-generated data or statistical predictions, and the possible 
decrease of contact between counselors and students. 

3. The research team should meet frequently with the counselors to obtain 
feedback on system problems so that necessary changes can be made. 

4. Counselors should improve their communication with each other, teachers, 
the administration, and especially with the students. The counselors 
should be trained in group and individual counseling techniques and 
would periodically be given the results of student questionnairs 
covering students' problems, feelings, and attitudes. 

5. Changes in counselors' attitudes and behavior should be assessed by 
collecting data from questionnaires and time logs, and the findings 
should be discussed with the counselors periodically. 



IV. RESULTS 

A. THE COMPUTER SYSTEM 

The exact workings of many parts of an information retrieval system depend on 
the structure chosen for the data base.* One cannot even flowchart a data 
base-generation program, for example, until one knows what files, tables, pointers , 
and so on, will be required — that is, how the data are to be organized. Thus the 
programming of an information retrieval system is generally not begun until a 
data-base structure is chosen. 

The problems involved in choosing or inventing a data-base structure for our 
system stemmed from the desire for rapid response on a very large data base. 

The system projected by our final report of lliase I ideally should have response 
times of a few seconds even for the most difficult queries. Such speeds require 
that the data base be "inverted" on every value of every item in the data base. 

This means that the data base must contain a list of all the cumulative records 
that have a value of "A12" for the item CIASS and another list of all cums that 



y. 

We use the term "data base" here to mean the data stored in the computer and 
the accompanying "bookkeeping" devices (tables, pointers, etc.) whose purpose is 
to facilitate the retrieval of data. 
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have a value of "B12" for CIASS, and so on for all existing values of CIASS. 
Similar lists must be provided for all other items in the data base; in fact, 
lists must be prepared for every item-value pair (such pairs are called keys ) 
that could conceivably be needed for a counselor's queries, The entire 
cumulative folder, except for anecdotal information, consists of such keys. 

Thus, this problem differs from the text-retrieval problem in which a large 
body of information (the texts) is stored, but inversion is done only for 
"Keywords," which, even if relatively numerous, will still be only a small 
fraction of the total data. 

To the best of our knowledge, no existing information retrieval system could 
give rapid response for the kind of data our counselors would be concerned with. 
Building such a system would have been a very big job, requiring considerable 
man-hours and computer time. Budgetary and manning problems militated against 
our following the obvious, logical process of designing the data base first. 
Certain parts of our information system, however, particularly those dealing 
with user input, are independent of data base. Consequently, it was possible 
to design, program, and check out seme system routines without a detailed data 
base design. Some of these parts were: 

• General input and output routines 

• Item definition 

• The Define function (and the associated Translate and Cancel functions) 

• Hie translation of the input query into an equivalent query couched 

entirely in primitives of the language 

• Checking of the grammar and punctuation of the query 

• Converting the population and the adjectives of the direct- object into 

conjunctive normal form 

Working on these routines first offered certain advantages. In the first place, 
it was possible to do the work with limited manpower and limited computer usage. 
Secondly, early development of these routines would make possible earlier 
counselor involvement with the system. The training of the counselors in the 
use of the language could be facilitated by processing their exercises through 
part of the system. Finally, these routines could be used with a simple, 
uninverted data-base structure, which, while slower in operation, would be 
implemented sooner. Eventual transition to an inverted data base would require 
little or no retraining of the counselors. 

When work on the computer system was discontinued, enough of the subsystems were 
implemented to give one a fairly good idea of what the total system would look 
like to the user. One could, for example, perform the following functions on-line 
describe a data-base (i.e., name and define items to be used), enter complicated 
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retrieval statements, and use the Define, Translate, and Cancel conmands. All 
user statements were decoded and checked for erros. However, because the data 
base structure had not been developed, storage and retrieval of data were not 
possible* 

B. THE HUMANISTIC APPROACH 

We felt that the humanistic approach taken in this project could effect desirable 
changes in the educational process. The results of this approach were very- 
encouraging. The counselors appeared to be changing their values in desired 
directions. They were concentrating less on data and more on students as human 
beings. They were concerned with the misuse of automation as well as with its 
advantages in efficiency. They seemed to believe that real, meaningful change 
could occur, and that they could help to bring it about. 

In summary, although the unavailability of funds prevented us from exploring 
fully the use cf computers in educational counseling, we did gain same prelimenary 
evidence that they can be used. We also feel their use must be carefully directed 
lest they be used to simply mechanize the shortcomings that already exist in the 
educational system. 
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SUM4AHY 



This final report describes the results of a study of how junior high school 
counselors' predictions of students' success in mathematics might be affected 
b *;hr counselors' knowing what success has been predicted for those students 
b) «. wjmputer. The purpose of the study was to throw some light on the general 
question of whether the "judgments" of a computer are viewed as possessing some 
unchallengeable authority over human judgments in similar situations. Ten 
experienced junior high school counselors were asked to predict the degrees of 
success in ninth-grade algebra of 100 anonymous eigbth-grade students on the 
basis of cumulative records and some additional information. The additional 
information included other predictions of the students' success in ninth-grade 
algebra. Seme of these predictions were clearly Identified as having been 
made by a computer; the remainder were prepared as if they had been made and 
recorded by another counselor. Both sets of predictions were further divided 
into optimistic and pessimistic predictions. Five treatments were then used: 

(1) cumulative record only, ( 2 ) cumulative record and an optimistic "computer" 
prediction, ( 3 ) cumulative record and pessimistic "computer" prediction, 

(4) cumulative record and an optimistic "hjad counselor" prediction, and 

( 5 ) cumulative record and a pessimistic "head counselor" prediction. 

After the counselor subjects had made their predictions, multiple linear 
regression analysis was performed on the prediction data to »«unirM> the dif- 
ferences among the five treatments. Riree null hypotheses were tested: (l) that 

there would be no differences in the subjects' predictions among the five 
treatments; ( 2 ) that there would be no differences in the subjects' predictions 
due to their having been given either optimistic or pessimistic predictions; and 
( 3 ) that there would be no differences among their predictions due to their 
having been given a computer's or a counselor's predictions. Hypotheses (l) and 

(2) were rejected. Hypothesis ( 3 ) was not rejected. The results showed that 
the subjects tended, in general, to make optimistic predictions, that they 
were swayed differently by optimistic and pessimistic predictions, but that 
they were not swayed differently by knowing that a computer or another counselor 
had made an earlier prediction. In general, then, we concluded that the notion 
that the computer exercises an unexamined authority, or sway, over counselors' 
use of their own experience and skills in making predictions is not borne out. 

The study did, however, raise some ancillary questions regarding the possibly 
undue influence of grading practices on students' scholastic careers. 

I. PROBLEM 

Research and development in the area of computer applications in secondary 
school counseling is growing rapidly. Although the computer can be helpful in 
counseling and guidance, however, it can also be misused, and its misuse can 
lead to undesirable consequences in the counseling operation. Therefore, 
while we should employ the resource of the computer for help in counseling, 
we should be sure that this resource is being properly used. 



